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Summary 


Ninth grade is a critical juncture for students—and can be a jarring transition. Even students 
a strong track record in the middle grades can experience academic difficulty, and those who enter 
high school with poor course grades, weak attendance, or behavior problems are especially at risk. 
An early misstep can have lasting implications: students who fall off track at the beginning of the 
school year may find it difficult to recover, and ninth graders who fail to earn the required number 
of academic credits are at elevated risk of dropping out. 


To support their most vulnerable students, high schools first need to identify them. This 
report uses eighth-grade student record data from two cohorts of first-time ninth graders (2015-16 
and 2016-17) in the School District of Philadelphia (SDP) to examine which student characteristics 
are most predictive of falling off track in ninth grade. With this information, schools can use middle 
grades data to identify their most at-risk ninth graders—and intervene to help them start strong in 
high school. 


KEY FINDINGS 


e Ninth-grade on-track rates varied by eighth-grade GPA, attendance, suspensions, and 
demographic characteristics. More than 90 percent of students who earned an A in any 
eighth grade core subject were on track at the end of ninth grade. Although earning a D or 
an F in a core subject was relatively uncommon, students who earned these grades were 
more likely to be off track than on track. Less than one-third of students with the weakest 
eighth grade attendance - those who attended less than 80 percent of the time, or missed at 
least seven weeks of school - were on track at the end of ninth grade. Few students (14.1 
percent) received an 8 grade out-of-school suspension, but of those who did, fewer than 50 
percent were on track at the end of ninth grade. None of the demographic characteristics 
predicted on-track rates as well as course grades, attendance, or suspensions. 


e Ofthe available student records information, eighth-grade core-subject GPA was the 
best predictor of ninth grade on-track status. This was followed by attendance and 
the number of out-of-school suspensions. However, knowing an eighth grade student’s 
attendance and number of suspensions provided only a small amount of additional 
information when predicting ninth grade performance. 


e Aset of four eighth-grade characteristics can identify about 30 percent of students 
who fall off track in ninth grade. These characteristics - what we are calling the Eighth 
Grade Predictors (8GPs) - were: (1) earning final course grades of D or F in all eighth grade 
courses; (2) failing a core subject; (3) attending school less than 80 percent of the time; and 
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(4) receiving two or more out-of-school suspensions in eighth grade. Sixty-eight percent of 
students with at least one 8GP fell off track in ninth grade. 


In all key student subgroups, at least 60 percent of students with at least one 8GP 
were off track at the end of ninth grade. For example, 70 percent of Black students and 
68.3 percent of White students with an 8GP were off track, but a smaller percentage of 
Asian students (60.9 percent) and Hispanic/Latinx students (64.2 percent) were off track. 
Likewise, 8GPs were less accurate for females than males: 61.9 percent of females with at 
least one 8GP were off track, compared to 72.8 percent of males. 


8GPs were more accurate for ninth grade students in some high schools than others 
but, in general, were more accurate for schools with higher percentages of off track 
students. At some schools the percentage of students with an 8GP who were off track was 
lower than 60 percent, while at other schools, 80 percent or more of those with an 8GP 
were Off track. 


High schools vary widely in the number of ninth graders with 8GPs. At 23 schools, less 
than 10 percent of the first-time ninth graders had any 8GP; at 22 schools, the figure was 
between 10 and 29 percent; and at 9 schools, the percentage was 30 or more. Schools with 
lowest percentages of ninth graders with an 8GP also tended to have the lowest percentages 
of ninth graders who fell off track to graduation. 


IMPLICATIONS FOR POLICY AND PRACTICE 


Because high schools face different percentages of students with 8GPs, they likely 
require different resources and strategies to help these students stay on track from 
the earliest days of high school. Schools with just one student entering with an 8GP may 
assign a guidance counselor or team of teachers to monitor that student’s attendance, 
behavior, and class performance. Other schools may need extra staffing resources 
(counselors, tutors, or social service providers) to help students improve their behavior, 
attendance, and homework habits, and to provide additional academic support. 
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Why this study 


Ninth grade marks a critical juncture in students’ education careers. When students transition 
to high school, they encounter more choices, expanded social opportunities, and increased 
academic challenges. While some students thrive in this new environment, others experience a 
degree of academic difficulty that places them off track to graduation by the end of ninth grade. Off- 
track students have not earned the expected number or type of course credits toward graduation, 
typically because they have enrolled in but not earned a passing grade in a required core subject 
course. Among two recent ninth-grade cohorts in the School District of Philadelphia (SDP), 
approximately one-third of the students were off track at the end of their first year in high school.1 


The transition to high school can be so jarring that even students who earned good grades 
and had good attendance in eighth grade fall off track to graduation. Research in Chicago and 
Philadelphia has found that, on average, ninth-grade outcomes do not just reflect how students did 
in school during their elementary and middle years. Stronger ninth-grade outcomes are associated 
with better odds of graduating from high school, independent of students’ performance in the 
middle grades.2 


However, it is also true that when some ninth graders enter high school, they already have a 
track record of poor course grades, weak attendance, and/or school behavior problems 
during the middle grades. These students are especially vulnerable to the challenges associated 
with the transition to high school. Without special attention from their new schools at the very start 
of ninth grade, students may fall off track early in the school year, setting a pattern from which it 
can be difficult to recover. 


To support these most vulnerable students as they start ninth grade, high schools first need 
a way to identify them. This involves two things: first, having access to students’ school record 
data from the middle grades and, second, knowing which student characteristics are most 
predictive of falling off track in ninth grade. Increasingly, high schools have ready access to data 
systems and dashboards that enable them to look at the prior school performance of new students. 
Less available are empirical analyses that pinpoint which aspects of students’ prior performance 
are the best clues that they will struggle in ninth grade.3 


according to the SDP’s 2018 Ninth Grade On-Track Definition, students who are on track at the end of ninth grade have earned at least 
(1) one course credit in each of the four core subjects (English, mathematics, social studies, and science), and (2) one additional credit 
from any source. More detail about the 2018 Ninth Grade On-Track Definition and its association with graduation rates is available in the 
following report: Theodore Wills, Defining 9" Grade Success: A New 9¢ Grade On Track Definition (Philadelphia: The School District of 
Philadelphia, 2018). Also see Molly Crofton and Ruth Curran Neild, Getting On Track to Graduation: Ninth Graders’ Credit Accumulation in 
the School District of Philadelphia, 2014-2016 (Philadelphia: The Philadelphia Education Research Consortium, 2018). 

2 Elaine M. Allensworth and John Q. Easton, The On-Track Indicator as a Predictor of High School Graduation (Chicago: Consortium on 
Chicago School Research, 2005); Ruth Curran Neild, Scott Stoner-Eby, and Frank Furstenberg, “Connecting Entrance and Departure 

The Transition to Ninth Grade and High School Dropout,” Education and Urban Society 40 no. 5 (2008), 543-569. 

3 Much of the research on early warning indicators has examined predictors of graduating or dropping out, rather than falling off track in 
ninth grade. For example, research on the middle grades predictors has examined the association between these characteristics and 
dropping out (Robert Balfanz, Liza Herzog, and Douglas Maclver, “Preventing Student Disengagement and Keeping Students on the 
Graduation Path in Urban Middle-Grades Schools: Early Identification and Effective Interventions,” Educational Psychologist 42 no. 4 
(2007), 223-235; Arthur Burke, Early identification of high school graduation outcomes in Oregon Leadership Network schools ((REL 
2015-079). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and 
Regional Assistance, Regional Educational Laboratory Northwest, 2015). However, we did identify two studies that examine the middle 
grades predictors of being on track in ninth grade: (1) Elaine M. Allensworth, Julia A. Gwynne, Paul Moore, and Marisa de la Torre, Middle 
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This study examined the student characteristics that are most predictive of falling off track 

during the first year of high school, using eighth grade data from two cohorts of ninth graders in 
the SDP. The study also shows how SDP high schools vary in the number of entering ninth graders 

at highest risk of falling off track and suggests different data-informed strategies that high schools 

could use to target available resources to the highest-risk ninth graders. 


What the study examined 


This study examines what can be known about entering ninth graders, based on their eighth grade 
school records, that can inform high schools about students’ likelihood of being on track at the end 
of their first year. Specifically, the study uses de-identified student record data for two cohorts of 
SDP ninth graders to identify the strongest predictors of being on track to graduation at the end of 
ninth grade. 


The research questions are: 


e What are the associations between key eighth grade student characteristics and being on 
track at the end of ninth grade? 

e Which eighth grade student characteristics, or combinations of characteristics, are most 
predictive of being on track at the end of ninth grade? 

e How do high schools differ in the percentage of students whose eighth grade records 
indicate that they are at highest risk for falling off track? 


To conduct these analyses, we examined eighth and ninth-grade data from SDP students who were 
first-time ninth graders in 2015-16 (the Class of 2019) or 2016-17 (the Class of 2020). Students in 
the analysis were enrolled in the SDP in both eighth and ninth grade and had eighth-grade 
attendance, suspension, test score, and course grade data, as well as data on ninth-grade credit 
accumulation. The sample includes 5,473 students from the Class of 2019 and 5,092 students from 
the Class of 2020, for a total of 10,565 students across both cohorts.* The study does not include 
information for students in Philadelphia’s charter high schools, which serve approximately 30 
percent of the students in grades 9-12 attending the city’s publicly-funded high schools. 


“To support these most vulnerable students as they start ninth 
grade, high schools first need a way to identify them.” 


Grade Indicators of Readiness in Chicago Public Schools (Chicago: Consortium on Chicago School Research, 2014) and (2) Michael J. 
Kieffer, William H. Marinell, and Nicky S. Stephenson, The Middle Grades Student Transitions Study Navigating the Middle Grades and 
Preparing Students for High School Graduation (New York: Research Alliance for New York City Schools, 2011). 

4 This study involves fewer students (n=10,565) than another PERC study on the percentage of ninth graders in the same two cohorts who 
were on track to graduation. That study, by Crofton and Neild (2018), had a sample size of 16,902. The present study includes only 
students who had eighth and ninth grade data from the two cohorts. Students who left the SDP after eighth grade or who entered the SDP 
in ninth grade are not included in this study. 
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To determine whether students were on track to graduate at the end of ninth grade, the study used 
the Ninth Grade On-Track Definition developed by the SDP in 2018. To make the analysis more 
relevant to current school improvement efforts, the analysis applies this definition retrospectively 
to the 2015-16 and 2016-17 cohorts, even though the definition was not developed until after these 
students had completed their first year in high school. According to this definition, students who 
are on track at the end of ninth grade have earned at least: 


e One course credit in each of the four core subjects (English, mathematics, social studies, and 
science), and 


e One additional credit from any source. 


More detail about the 2018 Ninth Grade On-Track Definition and its association with graduation 
rates is available in a report published by the SDP.5 


5 The 2018 On-Track Definition is different from the indicator displayed in the SDP’s School Progress Reports from 2017 and earlier. The 
previous on-track indicator required that students earn five credits during ninth grade without consideration of the subject area in which 
the credits were earned. Since high school students are required to obtain a certain number of credits in each subject in addition to the 
total credit accumulation, the indicator studied in this report provides a more complete picture of students’ progress towards graduation. 
For more information on the On-Track Definition, see Theodore Wills, Defining 9° Grade Success: A New 9t Grade On Track Definition 
(Philadelphia: The School District of Philadelphia, 2018). 
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How this study is similar to, and different from, 
research on early warning indicators of dropout 


The past 10 years have seen a rapid increase in the number of studies using student 
record data to identify “early warning indicators” of not completing high school. 
Researchers have identified early warning indicators for ninth graders and even for middle 
grades students. Typically, these predictors are combinations of attendance, behavior, 
and course grades. 


These studies examine factors associated with dropping out of high school, while this study 
explores factors associated with a different, but related outcome: falling off track in ninth 
grade. These studies are complementary but serve a different purpose. Studies of ninth 
grade predictors of dropout seek to raise awareness of how the transition to high school! 
affects the likelinood that students will earn a diploma — including changing the 
education prospects of students who appeared to be doing well before they entered 
high school. In contrast, this study seeks to identify students whose school performance in 
the middle grades signals that they are at high risk of falling off track in ninth grade — 
which, in turn, is linked to dropping out of high school. There is no need for school staff to 
wait and see how these students will do in high school: without intervention from the 
beginning of the school year, they are very likely to fall off track. 


Resources on early warning indicators of high school dropout include: 


e Robert Balfanz, Liza Herzog, and Douglas Maclver, “Preventing Student 
Disengagement and Keeping Students on the Graduation Path in Urban Middle- 
Grades Schools: Early Identification and Effective Interventions,” Educational 
Psychologist 42 no. 4 (2007), 223-235. 

e Elaine M. Allensworth and John Q. Easton, The On-Track Indicator as a Predictor of 
High Schoo! Graduation (Chicago: Consortium on Chicago School Research, 
2005. 

e Sarah Frazelle and Aisling Nagel, A Practitioner’s Guide to Implementing Early 
Warning Systems (REL 2015-056). (Washington, DC: U.S. Department of Education, 
Institute of Education Sciences, National Center for Education Evaluation and 
Regional Assistance, Regional Educational Laboratory Northwest, 2015). 
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Data and Variables 


This study used student-level data for first-time ninth graders in the SDP’s Classes of 2019 and 
« 2020. The data include no personally identifying information, and no individual student can 
= be identified. 


The study sample was limited to first-time ninth graders in 2015 or 2016 who were enrolled in 
at least one credit-bearing course in ninth grade. We further limited the sample to students 

= who were enrolled in eighth grade in SDP and who had valid data on eighth-grade course 

: grades, attendance, suspensions, and standardized test scores. Our initial data set included 
16,916 ninth graders across the two cohorts. Of these, 4,480 were not enrolled in the district 
during the previous year. An additional 1,871 students were enrolled in the district but were 

* missing state standardized test scores and/or course grades. We did not impute missing data 
for these 1,871 students because we had no evidence that the data were Missing at 
Random, an assumption of imputation methods. The final data set included 10,565 students. 


: Key variables are: 


On track status: The student would have been considered on track at the end of ninth 
= grade, according to the SDP’s 2018 Ninth Grade On Track Definition. 


Core subjects GPA: An average of the student's final grades in the four core subjects (eighth- 
grade English, mathematics, science, and social studies). 


Average Daily Attendance (ADA): The proportion of enrolled days in which the student 
attended school. 


Suspensions: The number of out-of-school suspensions in eighth grade. 


Test scores: Scores earned in eighth grade on the state-mandated English, mathematics, 
: and science standardized assessments (the Pennsylvania State System of Assessment, or 
PSSA). 


« Overage for grade: Students who were 15 years or older on September 1st of ninth grade. 
: English learner status: The SDP data included an indicator of whether a student was classified 
= as an English learner and receiving linguistic support. 


Low-income status: Whether a student's family received public assistance through Temporary 
» Assistance for Needy Families (TANF) and/or Supplemental Nutrition Assistance Program 
(SNAP). 


« Special education status: The data included an indicator of whether students were receiving 
special education services other than gifted and talented services. 


High school attended: The SDP data indicate the school each student attended for ninth 
= grade. For students who attended more than one school, we used the last school attended. 
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What the study found 


Eighth graders who earned higher final course grades in core subjects were more likely 
to be on track to graduation at the end of ninth grade. 


A student who is on track at the end of ninth grade, according to the SDP’s Ninth Grade On-Track 
Definition, has earned passing grades in at least five courses, including the four core subjects 
(English, mathematics, science, and social studies). It makes sense, then, that students with higher 
grades in core subjects in eighth grade were more likely to have earned passing grades in their 
ninth-grade courses and be on track (Table 1). 


Table 1. Ninth grade on-track rates by eighth grade final grades in core subjects 


Percentage of 
students with this 
grade who were on 
licefed @lamalialiame [Kee (=) 


Number of 
students earning 
saleme cele l=) 


Percentage of 
students earning 
saltme | cele (=) 


I Xeiiat=rankelicesMilnelme cote (=m Cae Laliale keke =) 


A 92.6% 2,975 24.4% 
B 79.6 3,231 30.6 
S 59.0 3,27] 31.0 
D 38.5 1,355 12.8 

F 21a) 133 1.3 
sake} amilare le Ike le (=m Carle laiiamel cere l=) 

A 92.5% 2,444 23.1% 
B 19.0 3,604 34.1 
Cc 56.9 3,251 30.8 
D 39.0 1,074 10.2 

F 21.4 192 1.8 
KYod (Ja lox-Milale le Kee (=m (=e lniame cerel=9) 

A 91.9% 2,672 25.3% 
B 79.0 3,440 32.6 
Cc 56.5 3,155 29.9 
D 38.3 1,141 10.8 

F 27.4 157 1.5 
RYeoKed (oF (Ufo (SM ilale le Ke Le (—m (=1elaiiame cole (=9) 

A 91.7% 2,992 28.3% 
B 76.0 3,509 33:2 
Cc 55.9 2,868 27.1 

D 37.9 1,024 9.7 

F 19.2 172 1.6 


Notes: n=10,565 


Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


Ready from the Start| 6 


@ 


Within each core subject, more than 90 percent of students who earned an A in eighth grade were 
on track at the end of ninth grade. For example, among students who earned an A in mathematics, 
92.6 percent were on track. At the other end of the grading scale, between 20 percent and 27 
percent of students who failed a core subject were on track. Although earning a D or an F ina core 
subject was relatively uncommon (ranging from 11 percent for social studies to 14 percent for 
mathematics), students with a grade in this range were more likely to be off track than on track at 
the end of ninth grade. 


Eighth graders with better attendance and fewer suspensions were more likely to be on 
track to graduation at the end of ninth grade. 


Among students who attended school at least 95 percent of the time in eighth grade (that is, those 
who missed no more than nine days of school), 80.9 percent were on track at the end of ninth grade 
(Table 2). For students with the weakest eighth grade attendance - those who attended less than 80 
percent of the time, missing at least seven weeks of school - less than one-third were on track. 


A minority of students (14.1 percent) had received an out-of-school suspension in eighth grade, but 
of those who did, fewer than 50 percent were on track at the end of ninth grade. For the small group 
of students receiving three or more suspensions, fewer than 30 percent were on track. 


These relationships are consistent with previous research in Philadelphia and elsewhere that 
demonstrates the association between attendance, behavior problems, and high school on-track 
rates.® 


Table 2. Ninth grade on-track rates by eighth grade attendance and suspensions 


Percentage of students 
i amialcmed alelce loa r=taty ites 
who were on track in 
raliauiame [cele {= 


Number of students Percentage of 
with this students with this 
characteristic characteristic 


idke) eXeyaicolamedime keh Ame liKcyare(=xem Cyc laliame LKecel=9) 


.95 or higher 80.9% 5,188 49.1% 
90 - .94 70.0 3,239 30.7 
85 - .89 93.5 1,143 10.8 
80 - .84 41.2 927 5.0 
Below .80 30.8 468 44 
Number of out-of-school suspensions (eighth grade) 

0 75.1% 9,077 85.9% 
1 48.0 902 8.5 
2 33.9 333 3.2 
3 or more 28.1 253 2.4 


Notes: n=10,565 


Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


6 Elaine M. Allensworth and John Q. Easton, The On-Track Indicator as a Predictor of High School Graduation (Chicago: Consortium on 
Chicago School Research, 2005); Ruth Curran Neild and Robert Balfanz, Unfulfilled Promise: The Dimensions and Characteristics of 
Philadelphia’s Dropout Crisis, 2000-2005 (Philadelphia: Philadelphia Youth Network, 2006). 
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Students with higher eighth grade state standardized test scores were more likely to be 
on track at the end of ninth grade, but more than half of the students at each 
proficiency level were on track. 


Among students who scored at the highest level (Advanced) on the eighth-grade mathematics, 
English, or science state standardized tests, between 89.2 percent and 94.3 percent were on track at 
the end of ninth grade (Table 3). However, students scoring at the lowest level (Below Basic) on any 
of the tests still had a better-than-even chance of being on track. Compared to the stark differences 
in on-track rates by course grades (Table 1) or attendance or suspensions (Table 2), the differences 
by PSSA proficiency levels are smaller. These descriptive results hint that, among the possible 
predictors of being on track in ninth grade, eighth grade test scores may not be the strongest 
indicators of whether a student will be on track. 


Table 3. Ninth grade on-track rates by eighth grade state standardized test scores 


Percentage of students 
co} Mi Lal SS YAW eo) Ke)i (ed (=Va lear 
level who were on 
Licefed @lamallaliame [xe le(=) 


Number of Percentage of 
students at this students at this 
proficiency level re) Ko} (ed (Va veaV al (=VA=| 


Mathematics PSSA (eighth grade) 


Advanced 94.3% 614 5.8% 
Proficient 84.3 1,232 11.7 
Basic 77.3 2,400 22.7 
Below Basic 62.7 6,319 59.8 
sare ame yam Coe Laliame lxele(=)) 

Advanced 89.2% 931 8.8% 
Proficient 79.1 3,156 29.9 
Basic 67.3 4,07] 38.5 
Below Basic 56.8 2,407 22.8 
AYod (=) alot SY. (ere Laliamelcelel=y) 

Advanced 89.5% 1,119 10.6% 
Proficient 79.0 2,157 20.4 
Basic 737 1,918 18.2 
Below Basic 61.8 5,371 50.8 


Notes: n=10,565 
Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 
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On-track rates differed by eighth-grade family income, race or ethnicity, gender, age, 
special education status, and English learner status. 


Lower percentages of male students, students receiving special education or English learner 
services, students who were overage for their grade, and students whose families received SNAP or 
TANF were on track at the end of ninth grade (Table 4). About 90 percent of Asian students were on 
track, compared to about 80 percent of White students and two-thirds of Black and Hispanic/Latinx 
students. However, in each of the groupings in Table 4, 50 percent or more of the students were on 
track to graduate. The differences in on-track rates were not as large for these characteristics as the 
differences associated with course grades, attendance, or suspensions. For example, the difference 
by English learner status was about one percentage point, while differences by special education 
status, gender, and family low-income status were about 10 percentage points. 


Table 4. Ninth grade on-track rates by demographics and eighth grade statuses 


Percentage of students 


iam ial cmed arelce loa r=vaty ies pila of pel etiade ot 
Sho Wore en hackin students with this students with this 
ninth grade characteristic characteristic 
Received services 61.1% 1,482 14.0% 
Did not receive services 719 9,083 86.0 
Received services 69.4% 744 7.0% 
Did not receive services 70.5 9,82] 93.0 
Female 74.6% 5,220 49.4% 
Male 66.2 
Age 
Overage in eighth grade 50.2% 687 6.5% 
Not overage 71.8 9,878 93.5 
Asian 90.8% 1,279 12.1 
Black 64.4 5,579 52.8% 
Hispanic/Latinx 65.2 1,840 17.4 
White 79.0 sis; 14.3 
Other 80.4 352 3.3 
Received SNAP or TANF 66.1% 6,609 62.6% 
Did not receive SNAP or TANF 71S 3,956 37.4 


Note: Some percentages do not sum to 100 because of rounding. n=10,565 
Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 
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Eighth grade GPA in core subjects was the best predictor of being on track. 


To be most useful to schools, the set of eighth-grade on-track predictors needs to be reduced to a 
small number of the most powerful predictors that can be used to target resources to students with 
the greatest risk of falling off track. To develop this smaller set of predictors, we used logistic 
regression, an advanced statistical technique, to summarize the unique and combined effects of the 
various eighth-grade characteristics described in Tables 1-4. Because each of the characteristics is 
measured in different scales and units, we standardized each variable into standard deviations 
from the mean. For test scores and course grades, standardized scores were calculated for each 
subject and then averaged into an overall score. 


Of all the student characteristics, eighth grade GPA in core subjects consistently had the largest 
association with being on track in ninth grade. When we looked at how well each characteristic, by 
itself, could explain being on track, we found that a key measure of association - the pseudo-R? - for 
GPA was .20 (Table 5). This means that eighth grade GPA explained 20 percent of the variation in 
being on track in ninth grade.” The characteristics with the next-highest pseudo-R? values were 
attendance (R2=.07), standardized test scores (R2=.06), and the number of suspensions (R2=.05).8 


Table 5. Summary of univariate logistic regressions of on-track status on eighth grade student 
characteristics, ordered by size of pseudo-R2 


Pseudo-R2 
GPA in core subjects 0.20 
Average daily attendance 0.07 
State standardized test scores 0.06 
Number of suspensions 0.05 
Asiant 
Blackt 
Hispanic/Latinxt a 
Other race or ethnicityt 
Overage for grade 0.01 
Family received SNAP and/or TANF 0.01 
Received special education services 0.01 
Female 0.01 
Received English learner services 0.001 


Notes: Regressions control for cohort. {Reference group is White. Reference group is male. *p<.001. n=10,565. 
Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


There is no hard-and-fast definition about what constitutes a large, medium, or small R2. However, 
generally speaking, in education research an R2 under .10 is considered to be a weak predictor, 
between .11 and .30 is considered to be a moderately good predictor, and above .30 is a very good 
predictor. Therefore, although there clearly are relationships between these characteristics and 
being on track, as shown descriptively in Tables 1-4, all characteristics except for GPA are relatively 
weak predictors. 


7 R2 values range from 0 to 1. An R2 of 1 means that the characteristic (or set of characteristics) explains, or fully accounts for, or perfectly 
predicts all of the variation in the outcome. An R? of O means that the characteristic does no better than the mean as a predictor. 
8 See Table A-1 for complete output from the univariate regressions. 
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Including a range of eighth grade characteristics improved the predictive model over 
GPA alone but not by much. 


We used multiple regression to understand whether the predictive power of GPA alone could be 
improved by adding other eighth grade characteristics available in student record data and 
examined descriptively above. Table 6 presents the results for the four characteristics with an R2 of 
.05 or above in Table 5, although the full set of characteristics was included in the model.? 


In this model, GPA, school attendance, and number of suspensions in eighth grade are statistically 
significant predictors of being on track. Standardized test scores is not a statistically significant 
predictor when these other characteristics are controlled (p=.101). 


The R2is .22, which represents a small improvement over using GPA alone (R2=.20) and is 
consistent with findings from previous research conducted in Chicago.1° This moderate-size R? tells 
us that it should be possible to identify some students who will fall off track in ninth grade on the 
basis of their eighth grade performance. At the same time, there is a great deal of variation (almost 
80 percent) in ninth-grade outcomes that is not explained or well-predicted by what we can know 
about students from their school records. In other words, while it is clear that we can predict being 
off-track for some students on the basis of eighth grade characteristics, it is also true that the ninth- 
grade transition can redirect students’ education careers in ways that could not have been 
predicted from prior performance. 


Table 6. Multivariate logistic regression of on-track status on eighth grade student 
characteristics 


Coefficient Ai felatelolcom-vice) 


GPA in core subjects ig -0.12 
Average daily attendance 026""* -0.03 
Number of suspensions -0.14*** -0.05 
State standardized test scores 0.15 -0.09 
Constant 1.33*** -0.21 
n 10,565 

Pseudo-r2 0.22 


Notes: ***p<.001. Regression controls for: Receipt of English learner services, receipt of special education services, overage for grade, 
race or ethnicity, gender, family low income status, and cohort. n=10,565. 
Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


9 See Table A-2 for a full set of logistic regression models. 
10 See R2 values presented on p. 31 of Elaine M. Allensworth, Julia A. Gwynne, Paul Moore, and Marisa de la Torre, Middle Grade 
Indicators of Readiness in Chicago Public Schools (Chicago: Consortium on Chicago School Research, 2014). 
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A set of four eighth grade characteristics can identify about 30 percent of students who 
fall off track in ninth grade. 


Focusing on the eighth grade characteristics that explained the most variance and were statistically 
significant (Tables 5 and 6), we examined different combinations of course grades, attendance, and 
numbers of suspensions to identify a small set of predictors of being off track that maximized both 
accuracy and yield. If a high percentage of students with a given characteristic fall off track in ninth 
grade, that characteristic is said to have high accuracy. Characteristics that identify a substantial 
proportion of students who fall off track have high yield. Accuracy and yield can sometimes be in 
tension: narrowly-defined predictors, while more accurate, can produce a lower yield. 


We identified four eighth grade characteristics that, individually and together, maximize 
accuracy and yield (Table 7).11 Students with any one of these four characteristics had about a 70 
percent chance of being off track in ninth grade. Twelve percent of all first-time ninth graders had 
at least one of these characteristics. 


Table 7. Eighth grade predictors of falling off track in ninth grade 


ACCURACY YIELD Number of Percentage of 
(% of students with C/o) Me} bs ike fet 4 ninth graders ninth graders 
ilaleiedalelrored Vi ifes students with this with this with this 
who were off track) fod sXe] ce Loa (VI) (69) vod ale] Ke led (=Variices vod ale] Ke Led (=Jariices 
Earned all Ds or Fs 77.1% 7.6% 310 2.9% 
poe cere 74.8 10.6 445 42 
course 
<80% attendance 69.2 10.4 468 4.4 
2+ suspensions 68.6 12.8 586 5.5 
eu oie saicu 68.7 28.7 1,305 12.4 
characteristics 


Notes: n=10,565 
Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


Not surprisingly, the two strongest predictors were related to course grades. The most 
accurate predictor was earning all Ds or Fs in eighth grade; 77.1 percent of the students who 
earned these grades were off track in ninth grade. Close behind in accuracy was failing an eighth 
grade core subject (English, mathematics, science, or social studies); about 75 percent of the 
students who failed one of these subjects were off track. Among students who had an eighth grade 
attendance rate of less than 80 percent or who had received two or more out-of-school 
suspensions, about 69 percent were off track. 


While there is some overlap in these categories (for example, there were some students who 
earned all Ds or Fs and failed a core course), the categories are not mere proxies for one another. 
Students who have two or more of these characteristics have an even higher risk of falling off track, 
ranging from 77.3 to 84.4 percent (Table 8). 


11 See Table A-3 for the full set of combinations that we tested. 
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Together, these four Eighth Grade Predictors - or 8GPs - identified almost 3 out of every 10 
off-track ninth graders. For individual characteristics, the yield - that is, the percentage of off- 
track students with that characteristic - ranged from 7.6 percent to 12.8 percent (Table 7). 
Importantly, these data tell us that high schools can identify, even before the first day of high school, 
about 30 percent of the students who will fall off track in ninth grade. Of course, identifying 
students at risk is not the same thing as developing an effective solution for each student’s 
particular set of challenges. But it is one of the first steps toward putting in place strategies to help 
students get on track to graduation. 


Table 8. Selected combinations of eighth grade predictors of falling off track in ninth grade 


ACCURACY YIELD Number of Percentage of 
(% of students with (% of off-track ninth graders ninth graders 


this characteristic students with this with this with this 
who were off track) characteristic) (od sre Koko (=i | om ed tle Ke Lea K=yariiles 


All Ds and Fs AND 
failed a core 
course in eighth 
grade 


Failed a core 
course AND 2+ 
suspensions in 
eighth grade 

All Ds and Fs AND 
<80% attendance 78.1% 1.8 73 0.7 
in eighth grade 
<80% attendance 
AND 2+ suspensions 77 3% 2.9 119 1.] 


in eighth grade 
Notes: n=10,565 
Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


84.4% 4.3% 160 1.5% 


81.2 2.3 127 1.2 


In all key student subgroups, at least 60 percent of students with at least one 8GP were 
off track at the end of ninth grade. 


We examined whether the accuracy and yield of the four 8GPs were similar for key subgroups of 
students (Table 9). This is an important analysis to ensure that the 8GPs do not perform so poorly 
for some groups of students that it would reduce the confidence with which schools could use 8GPs 
to identify students for support. 


We found that while the 8GPs were more accurate for some subgroups than others, at least 60 
percent of students in all subgroups with an 8GP were off track at the end of ninth grade. For 
example, 70 percent of Black students and 68.3 percent of White students with an 8GP were off 
track, but a smaller percentage of Asian students (60.9 percent) and Hispanic/Latinx students (64.2 
percent) were off track. Likewise, 8GPs were less accurate for females (61.9 percent) than males 
(72.8 percent). 


These differences may suggest that, for some groups of students, it is more difficult to identify who 
will fall off track on the basis of eighth grade characteristics. Alternatively, it may be the case that 
other sets of eighth-grade predictors would be more accurate for these students. The benefits of 
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greater accuracy for some student groups, however, would need to be weighed against the 
drawbacks of reducing the simplicity and ease of use of a single set of EPGs to use for all students. 


Table 9. How the Eighth Grade Predictors (8GPs) perform for key subgroups 


ACCURACY YIELD 
C/o} Se) IES [Te (Ta (% of off-track students 
who were off track with an 8GP) 


KY oY-Yerfe] =Xe [Uroxe] f(e) axa (ox=sm (=e Laliame Keke l=9) 


Received services 


70.5% 


26.5% 


Did not receive services 


63.9 


38.4 


ake LAM cexel dal=ymex=y Adcox Colle Laliame xe le{=)) 


Received services 


98.8% 


21.9% 


Did not receive services 


69.4 


29.2 


Xo fox- me) m= alalediavg 


Asian 60.9% 11.9% 
Black 70.0 32.3 
Hispanic /Latinx 64.2 21.8 
White 68.3 26.4 
Other race/ethnicity 69.2 27.3 
Gender 

Female 61.9% 22.8% 
Male 72.8 33.0 


Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


At some high schools, no first-time ninth graders have any of the Eighth Grade 
Predictors (8GPs) of falling off track. At other schools, 30 percent or more have at least 
one predictor. 


Just as high schools vary in the percentage of ninth graders who are on track, they also have 
different percentages of students with a record of low grades or course failure, poor attendance, or 
suspensions in eighth grade. At 23 schools, less than 10 percent of the first-time ninth graders had 
any 8GP; at 22 schools, the figure was between 10 and 29 percent; and at 9 schools, the percentage 
was 30 or more (Table 10). Schools with lowest percentages of ninth graders with an 8GP also 
tended to have the lowest percentages of ninth graders who fell off track to graduation.!2 


These percentages tell us that Philadelphia high schools face very different levels of challenge in 
planning interventions to keep students with known risk factors on track in ninth grade. The 
numbers of students with 8GPs, in addition to the percentages, make clear the different challenges 
that schools face in addressing student need from the very beginning of high school. For the group 
of 23 schools that had the lowest percentage of ninth graders with an 8GP, the median number of 
students with an 8GP was just two students. In contrast, for schools where more than 30 percent of 
the students had an 8GP, the median was 25 students. 


12 Molly Crofton and Ruth Curran Neild, Getting On Track to Graduation: Ninth Graders’ Credit Accumulation in the School District of 
Philadelphia, 2014-2016 (Philadelphia: The Philadelphia Education Research Consortium, 2018). 
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Table 10. Percentage of first-time ninth graders with an Eighth Grade Predictor (EPG), by high 


school 


Academy at Palumbo 
Arts Academy-Rush 
Bodine 

C.A.P.A. 

Central 

Constitution 
Engineering & Science 
Franklin Learning Ctr 
G.A.M.P. 

Girls 

HS of Future 
Hill-Freedman 
Lankenau 
Masterman 
Northeast 

Parkway CC 
Parkway NW 
Parkway West 

Phila Military 

Saul 

Sci. Lead. Academy 
SLA-Beeber 
Swenson 


PNavaltro| malanlox-ymeo) M=Va) (=) 


Furness 
Kensington CAPA 
Kensington Health 
Lincoln 
Mastbaum 
Motivation 

Penn Treaty 
Randolph 
Robeson 

The Linc 
Washington 
Workshop School 


flake Mallauiamelcore( eM iia 


Building 21 

Dobbins 

Edison 

Kensington 
Kensington Business 
King 

Roxborough 

South Phila 

The U School 

West Phila 


Bartram 

Fels 

Frankford 

Franklin 

Kensington Ed. 
Overbrook 

Phila Virtual 

Sayre 

Strawberry Mansion 


rol aN Aco)im iat= Mm colelm ol cto | (oa Koy ecm C=Xellolamelarecelate |= 


2 students 
(0-44) 


10 students 
(6-39) 


17 students 
(5-38) 


25 students 
(5-41) 


Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


8GPs were more accurate for ninth graders attending some high schools than others 
but, in general, were more accurate for ninth graders attending schools with higher 
percentages of off-track students. 


On average across the SDP, 68.7 percent of students with an 8GP were off track at the end of ninth 
grade. At some schools, however, the percentage of students with an 8GP who were off track was 
lower than 60 percent, while at other schools, 80 percent or more of those with an 8GP were off 


track (Table 11). 


Generally speaking, the schools at which 8GPs were the least accurate (<60 percent) had lower 
percentages of ninth graders with an 8GP (compare Table 10) and fewer off-track students.13 In 
other words, in the 8GPs were more accurate in schools with higher off track rates and more 
students who enter ninth grade with an 8GP. These are the schools where it is most important to 
have 8GPs perform best, since these are the schools that struggle most with keeping students on 


track. 


13 Molly Crofton and Ruth Curran Neild, Getting On Track to Graduation: Ninth Graders’ Credit Accumulation in the School District of 
Philadelphia, 2014-2016 (Philadelphia: The Philadelphia Education Research Consortium, 2018). 
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Table 11. Accuracy of the Eighth Grade Predictors (8GPs), by high school 


Percentage of ninth graders with any 8GP who were off track 


At these schools, <60% 


of students with an 8GP 


At these schools, 
80%+ of students with 


were off track 
Academy at Palumbo 
Arts Academy-Rush 
C.A.P.A, 
Central 
Dobbins 
Fels 
Franklin Learning Ctr 
G.A.M.P. 
HS of Future 
Hill-Freedman 
Lankenau 
Masterman Motivation 
Parkway NW 
Parkway West 
Penn Treaty 
Phila Military 
Phila Virtual 
Roxborough 
Sayre 
SLA-Beeber 
The Linc 


Bodine 
Constitution 
Engineering & 
Science Frankford 
Kensington 
Kensington CAPA 
Kensington Ed. 
King 

Randolph 
Robeson 

Saul 

South Phila 
Strawberry Mansion 
Swenson 


Bartram 

Edison 

Girls 

Kensington Business 
Mastbaum 
Overbrook 
Washington 

West Phila 


an 8GP were off track 
Building 21 

Franklin 

Furness 

Kensington Health 
Lincoln 

Northeast 

Parkway CC 

Sci. Lead. Academy 
The U School 
Workshop School 


Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


Implications for policy and practice 


Because high schools differ in the number and percentage of first-time ninth graders 
with an 8GP, they also differ in the resources and strategies needed to help these 
students stay on track from the earliest days of high school. 


Knowing how schools differ in the percentages and numbers of students with an 8GP provides 
insight into the resources and strategies needed by different schools to support these vulnerable 
students. For schools with just one student entering with an 8GP, the intervention at the start of 
high school may be ongoing monitoring of attendance, behavior, and class performance by a 
guidance counselor or team of teachers. For schools with a moderate number of students with a 
8GP (for example, 10-15 students), it might be possible to assign each student a teacher-advocate 
who is responsible for regular check-ins with the student and his or her teachers, counselor, and 
parent. Larger high schools, with more staff, might be able to take this approach even if they have 
more students. Schools of small or moderate size, however, will very likely need extra staffing 
resources (counselors, tutors, or social service providers) to help students improve their behavior, 
attendance, and homework habits, and to obtain any additional support needed to improve 


academic skills. 
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Schools serving middle grades students can use these indicators to identify 
students who need interventions before starting high school. 


This report has showed how some students enter ninth grade with a track record of academic 
difficulty , behavioral challenges, or poor attendance. These problems carry over into ninth grade, 
placing students at heightened risk of dropping out of school. Schools serving these middle grades 
students may already have targeted many of these students for special intervention during eighth 
grade, but the findings in this report suggest a particular urgency for identifying and implementing 
the educational, social, and/or health services that will help these students get on a new path. 
Interventions do not need to wait until high school; the sooner steps are taken to address these 
students’ problems, the more opportunities there are for a fresh start. 


Ready from the Start | 17 


Appendix A: Additional tables 


Table A-1. Univariate logistic regressions of on-track status on various eighth grade factors 


a Oe ae (3) (4) (5) (6) (7) (8) (9) 

Asian 0.961*** 

(0.252) 
Black -0.731*** 

(0.228) 
Hispanic/ -0.699%** 
Latinx 

(0.213) 
Other race 0.0794 
or ethnicity 

(0.228) 
Overage -0.924*** 

(0.130) 
Female 0.405*** 
(0.0975) 
English -0.0560 
learner 
(0.171) 
Special -0.485*** 
education 
(0.152) 
Low-income -0.570*** 
(0.110) 
Attendance 0.676*** 
(0.0646) 
Test Scores 0.687*** 
(0.113) 
Suspensions -0.553*** 
(0.0819) 
Course Beoo2te* 
grades 
(0.143) 

Class of 2020 0.178* 0.180* 0.185* 0.182* 0.180* 0.189* 0.140 0.192 0.185* 0.204 

(0.107) (0.104) (0.104) (0.103) (0.104) (0.104) (0.115) (0.131) (0.110) (0.128) 
Constant 1.241*** | 0.848*** | 0.586*** | 0.783*** | 0.853*** 1.148*** | 0.847*** | 0.860*** | 0.791*** 1.045*** 

(0.260) (0.167) (0.161) (0.171) (0.173) (0.200) (0.154) (0.160) (0.155) (0.148) 
Obs 10,565 10,565 10,565 10,565 10,565 10,565 10,565 10,565 10,565 10,565 
Pseudo R2 0.0407 0.0116 0.00835 0.00144 0.00665 0.0137 0.0742 0.0571 0.0459 0.195 


Robust standard errors in parentheses 


Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 


*** n<0.01, ** p<0.05, *p<0.1 
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Table A-2. Multivariate logistic regressions of on-track status on various eighth grade factors 


eee Academic Academic eee olis 
TNexeleol—Tnal(es Behavior | Demographics iz: + + elicd 
Spec Ed Behavior pY-TaateYel ce] e)al(es HSpecied 


Low-income -0.481*** -0.244*** -0.185*** 
(0.0933) (0.0701) (0.0702) 
Asian 1.006*** 0.214 0.155 
(0.250) (0.199) (0.192) 
Black -0.655*** -0.242 -0.256* 
(0.206) (0.156) (0.155) 
Hispanic/Latinx -0.619*** -0.374** -0.428*** 
(0.188) (0.147) (0.152) 
Other race or 0.0875 0.0677 0.0549 
ethnicity 
(0.218) (0.204) (0.194) 
Female 0.435*** -0.0818 -0.0144 
(0.0954) (0.0770) (0.0832) 
Overage “0: 752*** -0.318*** -0.306*** 
(0.0995) (0.0799) (0.0850) 
Course Grades L313* 1.164*** 1.2798" 1.351** 1.170*** 
(0.126) (0.122) (0.118) (0.130) (0.120) 
Test Scores 0.207** 0.168* 0.141 0.245*** 0.146 
(0.0932) (0.0914) (0.0898) (0.0937) (0.0891) 
Attendance 0.564*** 0.263*** 0.256*** 
(0.0562) (0.0327) (0.0326) 
Suspensions -0.363*** -0.138** -0.136*** 
(0.0724) (0.0537) (0.0528) 
English learner -0.0297 0.242 0.159 
(0.162) (0.158) (0.174) 
Special - 0.406*** 0.453*** 
education 0.484** 
(0.149) (0.103) (0.107) 
Class of 2020 0.243* 0.147 0.188* 0.180* 0.215 0.229 0.250* 0.212 
(0.143) (0.118) (0.110) (0.104) (0.146) (0.144) (0.145) (0.147) 
Constant 1.044*** 0.843*** 1.338*** 0.855** 1.041*** 1.451*** 0.973*** 1325" 
(0.145) (0.150) (0.274) (0.180) (0.144) (0.213) (0.147) (0.208) 
Observations 10,565 10,565 10,565 10,565 10,565 10,565 10,565 10,565 
Pseudo R2 0.198 0.0902 0.0642 0.00666 0.211 0.204 0.201 0.220 
Robust standard errors in parentheses 
*** D<0.01, ** p<0.05, * p<0.1 


Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 
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Table A-3. Accuracy of eighth grade predictors, individually and in combination 


ACCURACY Number of students 


Percentage of students with this 


Lam Lalemed are Kees t= Viv ile 


‘al t e ti 
who were off track fed po] Go Fea (1113 I [es 


Earned no As or Bs 59.9% 2352 
Earned Ds or Fs only 77.1 310 
Failed a core course 74.8 445 
Attendance <90% 54.5 2138 
Attendance <85% 63.7 995 
Attendance <80% 69.2 468 
Suspended 1+ times 58.5 1488 
Suspended 2+ times 68.6 086 
Suspended 3+ times 719 253 
No As or Bs and attendance <90% 70.5 980 
No As or Bs and suspended 1+ times 73.9 767 
Attended <90% and suspended 69.1 740 
No As or Bs or attendance <90% 93.6 3510 
No As or Bs or suspended 1+ times 5o.7 3073 
Attended <90% or suspended 52.9 2886 


Note: n=10,565 
Source: Authors’ calculations based on student data provided by the School District of Philadelphia (2015-16, 2016-17, and 2017-18). 
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